Detectron2
This is also an updated version of my previous blog detectron2 published on October 13, 2019. I lost the previous source for this blog, but, let’s re-write it.
The autumn in both Vancouver and Seattle is gorgeous…
| Overview Seattle on Space Needle | Around Space Needle |
|---|---|
 |
 |
| Overview Seattle on Space Needle | Around Space Needle |
 |
 |
| Overview Seattle on Space Needle | Around Space Needle |
 |
 |
| Overview Seattle on Space Needle | Around Space Needle |
 |
 |
Alright, let’s rapidly test Detectron2.
Installation is detailedly summarized in INSTALL.md.
We can simply follow GETTING_STARTED.md for some simple demonstrations. Make sure you’ve downloaded the demo pictures from Detectron1 demo and save under Detectron2’s folder demo.
1 | ➜ detectron2 git:(master) python demo/demo.py --config-file configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml \ |
Let’s take a look at the result:
And of course, my pictures taken in Seattle and Vancouver:
| Overview Seattle on Space Needle | Around Space Needle |
|---|---|
 |
 |
| Overview Seattle on Space Needle | Around Space Needle |
 |
 |
| Overview Seattle on Space Needle | Around Space Needle |
 |
 |
| Overview Seattle on Space Needle | Around Space Needle |
 |
 |
Tensorflow 2.X
This is actally an updated version of my previous blog Tensorflow 2.0 published on October 12, 2019. What’s unfortunate is: I lost the source of that previous blog. What’s fortunate is: I have my Tensorflow updated from 2.0 to 2.1. Anyway, let’s begin:
- Previous blog: Flying back to China soon. Before leaving, here comes a simple blog testing Tensorflow 2.0. In this blog, I strictly follow Amita Kapoor and Ajit Jaokar’s free book Getting Started with TensorFlow 2.0.
1. Check GPU
1 | ➜ ~ python |
Then, for simplicity, let’s try it out directly:
2. Tensorflow 2.X
3. Tensorflow Dataset
Corona Virus
Hello everyone… As you all know that currently, my hometown Wuhan, China is suffering the MOST SERIOUS infectious disease - Corona Virus in her history. My parents are still in this locked city. People have been suffering the threaten from the death for over 20 days. I would have to write something, and WISH ALL MY BEST to my beloved parents, and all my country fellows who are still in Wuhan.
In this blog, all data are obtained from WHO Corona Virus Reports.
And, all plots are generated by altair.
So far, I can ONLY make that much…
| Confirmed Cases In China | Death Over The World |
|---|---|
 |
 |
God bless Dr. Li WenLiang, and shame on rubbish in Chinese government.
Updated on:
- February 10, 2020
- February 14, 2020
- February 15, 2020
- February 16, 2020
- February 17, 2020
- February 19, 2020
- February 21, 2020
So far, the data is still lack of info such as age, gender, etc. Let’s keep an eye on Corona Virus.
HiSilicon Hikey 970 - Shunya Desktop Image For HiKey970
1 | ➜ binaries sudo ./hikey_idt -c config |
Build Toolchain Using Crosstool-NG
Today, we’re going to build our own toolchain using crosstool-NG. There are so many cases that we want to build a particular toolchain for a specific embedded system development board. One MOST important reason for that is probably because the specific development board has limited resource to build upon, therefore, the time-consumption to build some software on the development board is VERY inefficient. In the other way around, cross compiling on the host PC is MORE efficient with respective to the time-consumption.
In this blog, for simplicity, we take Raspberry Pi 3B as our demo development board, for which we are building the cross compiler. Some references can be found at:
1. Installation
How to install crosstool-NG are thoroughly summarized at its official website. In my case, I had it installed under folder /opt/compilers/crosstool-ng. Let’s take a look:
1 | pwd |
And make sure /opt/compilers/crosstool-ng/bin is under environment variable PATH.
2. Configuration
Under any directory that you want to save your .config file, we can configure our target cross compiler.
1 | ct-ng menuconfig |
According to elinux: RPi Linaro GCC Compilation, we need to do the following selections:
-
Paths and misc options:
- Try features marked as EXPERIMENTAL: ticked
- Prefix directory: input the full path where you want to save the built toolchains
- Number of parallel jobs: 4. Sorry that I’ve NO idea if this is the number of cores on Raspberry Pi 3B, but Raspberry Pi 3B does have 4 cores.
-
Target options:
- Target Architecture: arm
- Default instruction set mode: arm
- Use EABI: ticked
- append hf to the tuple (EXPERIMENTAL): ticked
- Endianness: Little endian
- Bitness: 32-bit
- Emit assembly for CPU: cortex-a53
- Use specific FPU: vfp
- Floating point: hardware (FPU)
-
Toolchain options:
- Tuple’s vendor string: rpi
-
Operating System
- Target OS: linux
- Version of Linux: 4.20.8. Currently, I flashed Raspbian Buster with desktop and recommended software, 2019-07-10 onto my Raspberry Pi 3B, it comes with kernel of version 4.19.66, which is NOT in the list. That’s why I decided to select 4.20.8 and try my luck.
😝1
2➜ ~ uname -a
Linux raspberrypi 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux -
Binary utilities:
- Binary format: ELF
- Version of binutils: 2.31.1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19➜ ~ apt show binutils
Package: binutils
Version: 2.31.1-16+rpi1
Priority: optional
Section: devel
Maintainer: Matthias Klose <doko@debian.org>
Installed-Size: 95.2 kB
Provides: binutils-gold, elf-binutils
Depends: binutils-common (= 2.31.1-16+rpi1), libbinutils (= 2.31.1-16+rpi1), binutils-arm-linux-gnueabihf (= 2.31.1-16+rpi1)
Suggests: binutils-doc (>= 2.31.1-16+rpi1)
Conflicts: binutils-multiarch (<< 2.27-8), modutils (<< 2.4.19-1)
Homepage: https://www.gnu.org/software/binutils/
Download-Size: 56.9 kB
APT-Manual-Installed: no
APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
Description: GNU assembler, linker and binary utilities
The programs in this package are used to assemble, link and manipulate
binary and object files. They may be used in conjunction with a compiler
and various libraries to build programs. -
C-library:
- C library: glibc
- Version of glibc: 2.28
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25➜ ~ apt show libc-bin
Package: libc-bin
Version: 2.28-10+rpi1
Priority: required
Essential: yes
Section: libs
Source: glibc
Maintainer: GNU Libc Maintainers <debian-glibc@lists.debian.org>
Installed-Size: 3,015 kB
Depends: libc6 (>> 2.28), libc6 (<< 2.29)
Recommends: manpages
Homepage: https://www.gnu.org/software/libc/libc.html
Download-Size: 657 kB
APT-Manual-Installed: yes
APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
Description: GNU C Library: Binaries
This package contains utility programs related to the GNU C Library.
.
* catchsegv: catch segmentation faults in programs
* getconf: query system configuration variables
* getent: get entries from administrative databases
* iconv, iconvconfig: convert between character encodings
* ldd, ldconfig: print/configure shared library dependencies
* locale, localedef: show/generate locale definitions
* tzselect, zdump, zic: select/dump/compile time zones -
C compiler:
- Show gcc versions from: GNU
- Version of gcc: 8.3.0
- gcc extra config: --with-float=hard
- Link libstdc++ statically into the gcc binary: tick
- C++: tick
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20➜ ~ apt show gcc
Package: gcc
Version: 4:8.3.0-1+rpi2
Priority: optional
Section: devel
Source: gcc-defaults (1.181+rpi2)
Maintainer: Debian GCC Maintainers <debian-gcc@lists.debian.org>
Installed-Size: 46.1 kB
Provides: c-compiler, gcc-arm-linux-gnueabihf (= 4:8.3.0-1+rpi2)
Depends: cpp (= 4:8.3.0-1+rpi2), gcc-8 (>= 8.3.0-1~)
Recommends: libc6-dev | libc-dev
Suggests: gcc-multilib, make, manpages-dev, autoconf, automake, libtool, flex, bison, gdb, gcc-doc
Conflicts: gcc-doc (<< 1:2.95.3)
Download-Size: 5,200 B
APT-Manual-Installed: no
APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
Description: GNU C compiler
This is the GNU C compiler, a fairly portable optimizing compiler for C.
.
This is a dependency package providing the default GNU C compiler.
After we save the configuration to file .config, we Exit the crosstool-NG configuration dialog.
3. Build
Please remember to:
unset LD_LIBRARY_PATHbefore building. Otherwise, you’ll meet some ERROR messages.mkdir ~/srcbefore building. Otherwise, whenever you tried to rerunct-ng build, you’ll have to download ALL required packages from scratch.
1 | ct-ng build |
This process may take a while. I’m going to sleep tonight. Continue tomorrow…
Alright, let’s continue today. Glad to know it’s built successfully.
😄
Let’s FIRST take a look at what’s under the current folder.
1 | ls |
And then, let’s take a look at what’s built under the specified destination folder.
1 | ls ~/....../CrossCompile/RPi |
Finally, let’s take a look at the version of our built cross compilers for Raspberry Pi 3B.
1 | arm-rpi-linux-gnueabihf-gcc --version |
Additional issue: It seems current crosstool-NG does NOT officially support Python. Please refer to my issue at crosstool-NG.
4. Compile/Build a Package with Generated Toolchain
For most of the packages nowadays, they are
- either supported by make:
./configure -> make -> make install - or supported by CMake:
mkdir build -> cd build -> ccmake ../ -> make -> make install
How to use our generated toolchain to compile/build our target packages?
- For the FIRST option, you can follow the crosstool-NG Using the toolchain.
- For the SECOND option, you are welcome to follow CMake Toolchains.
Today, we’re going to take the package flann as an example, which are to be built with CMake.
4.1 CMakeLists.txt Modification
4.1.1 CMake Toolchains - Cross Compiling for Linux
By following CMake Toolchains - Cross Compiling for Linux, we FIRST modify flann CMakeLists.txt a bit by adding the following lines before project(flann).
1 | cmake_minimum_required(VERSION 2.6) |
- CMAKE_SYSROOT specify the sysroot directory which emulate your target environment, here, Raspberry Pi 3B
- CMAKE_STAGING_PREFIX is where the STAGING results store, for the reason that final results may require to be built in multiple stages. You may refer to Linux From Scratch for further background knowledge about that.
- tools actually specify the building tool directory. Making sure all generated cross compiling tools are under folder
${tools}/bin.
4.1.2 Ignore hdf5
In addition, for the emulated Raspberry Pi 3B sysroot, hdf5 is NOT supported. Therefore, let’s simply comment out the following line in flann CMakeLists.txt.
1 | #find_hdf5() |
4.2 Cross Compile
Now, let’s start cross-compiling flann.
1 | mkdir build |
and press c and then t, you’ll see the cross compiling toolchains have been automatically configured as:
1 | BUILD_CUDA_LIB *OFF |
The LAST step before make is to modify some of the parameters accordingly, in my case:
- BUILD_PYTHON_BINDINGS: ON -> OFF
- CMAKE_CXX_FLAGS:
-I/usr/include(for lz4.h from my host Ubuntu 19.04) - CMAKE_C_FLAGS:
-I/usr/include(for lz4.h from my host Ubuntu 19.04) - CMAKE_VERBOSE_MAKEFILE: OFF -> ON
- CMAKE_INSTALL_PREFIX:
/usr/local/-> under which directory you want to install, which can be IGNORED for now
Now, press c, you’ll see:
1 | CMake Warning at CMakeLists.txt:99 (message): |
A warining of missing hdf5 is clearly reasonable and acceptable. Then press g.
Finally, it’s the time for us to cross build flann.
1 | make -j8 |
VERY Important:
- If you build flann from source on a Raspberry Pi 3B, your system is going to hang HERE, possibly due to lack of memory…
- Raspberry Pi 3B has 4 cores ONLY, but now you can use MORE cores on your host PC, which is clearly one advantage of using cross compiling.
Finally, you’ll see flann has been successfully cross compiled as:
1 | ....../flann/src/cpp/flann/algorithms/autotuned_index.h: In member function 'void flann::AutotunedIndex<Distance>::optimizeKMeans(std::vector<flann::AutotunedIndex<Distance>::CostData>&) [with Distance = flann::L2<double>]': |
You can now:
make installto install the built/generated libraries installed under CMAKE_INSTALL_PREFIX- copy and paste the built/generated libraries onto Raspberry Pi 3B and use it directly.
BTW: do NOT forget to install the header files.
5. Additional Issues
Multilib/multiarch seems to be problematic nowadays. Please pay attention to Multilib/multiarch. Some related issues are enumuated as the end of this blog.
- Multilib caveats from Official Notes on specific toolchain features
- fatal error: gnu/stubs-hard.h: No such file or directory from crosstool-NG
- fatal error: gnu/stubs-hard.h: No such file or directory from gcc-cross-compiler
- Build error: “features.h: No such file or directory” from risc-v tools
- crosstool-NG risc-v linux multilib issue
- raspberrypi toolchain issue
Alright, that’s all for today. Let me go to bed. Good bye…
😪
😴
Stereo Vision on VCSBC nano Z-RH-2 - PART I
Today, we are going to talk about a fabulous project: stereo vision on a zynq-7010 board.
1. VCSBC nano Z-RH-2
1.1 Hardware
We are using a VCSBC nano Z-RH-2 board for today’s experiment. The board adopted looks like the following:
| Front | Back | Connector |
|---|---|---|
 |
 |
 |
More detailed specifications, please refer to Vision Components’s official website.
1.2 Software
After you set up a static IP for this Vision Components SBC, it’s pretty straightforward to ssh into the system.
1 | ssh user@192.168.1.79 |
Currently, VC provides Linux Kernel 3.14.79.
1 | user@VC-Z:~$ uname -a |
And, let’s take a look at the dual ARMv7 CPUs on zynq-7010.
1 | user@VC-Z:~$ cat /proc/cpuinfo |
2. Stereo Vision
Sorry everybody. Today, I ONLY test stereo vision on ARM. I’ll try to figure out how to flash an open source IP of stereo vision onto zynq-7010, or write my own ASAP.
Hmmmmmmmm… It’s better I keep my code in dark???
😝
2.1 Classical Image Pairs
Let’s try out the stereo vision on some .pgm image pairs FIRST.
1 | user@VC-Z:~/longervision$ ./pgmpair ../images/aloe_left.pgm ../images/aloe_right.pgm |
My GOD… It’s UNBELIEVABLY SLOW.
| aloe_left | aloe_right |
|---|---|
 |
 |
| aloe_left Stereo | aloe_right Stereo |
 |
 |
2.2 Live Video Pairs
The BEST demo code to test Vision Componenets Stereo Vision is Eclipse_Example_Projects_VC_Z.
2.2.1 imageCaptureTest
1 | ./imageCaptureTest |
2.2.2 imageCaptFPS
1 | 03:41:47[root@VC-Z] /home/user/vc/Eclipse_Example_Projects_VC_Z/imageCaptFPS |
The above 2 examples are directly run on Vision Components’s board without display, for the given cable is of VGA connector, which is ALREADY outdated for many years. Therefore, in order to show the captured image pairs, in the next section, we’ll have to stream the captured data to a host computer, and display the real-time video pairs.
2.3 Stream/Display From Host Computer
Due to Vision Componenets’ Official documentation VCLinux_Getting_Started.pdf, images captured from camera can be displayed on a remote hosting PC.
2.3.1 Eclipse_Example_Projects_VC_Z (Not Preferred)
Eclipse_Example_Projects_VC_Z.zip provides some source code, which displays images captured from camera on the hosting PC’s Eclipse, by adding the camera as a Remote System to Eclipse.
Anyway, by using this method, you need to prepare ALL the following software and packages in advance.
- Eclipse
- CDT: install by Check for Updates with p2 software repository: http://download.eclipse.org/tools/cdt/releases/9.8
- Direct Remote C++ Debugging: installed from Eclipse Marketplace
2.3.2 vcimgnetclient.py
Besides the above method, a much more straightforward method is to adopt python script vcimgnetclient.py provided by Vision Components. However, vcimgnetclient.py is ONLY python2 compatible, and Vision Components has NO plan to provide a python3 compatible version of vcimgnetclient.py.
Therefore, the KEY to use the 2nd method is to make vcimgnetclient.py python3 compatible.
2.3.2.1 2to3
1 | 2to3 vcimgnetclient.py |
2.3.2.2 pygi-convert
Please refer to General Porting Tips
1 | ./pygi-convert.sh vcimgnetclient.py |
2.3.2.3 Try Running
1 | python vcimgnetclient.py |
2.3.2.4
For ALL vbox.pack_start, add one parameter 0 at the end for each case. For instance:
vbox.pack_start(self.menu_bar, False, False)
to
vbox.pack_start(self.menu_bar, False, False, 0)
Oh my god … There are STILL SO MANY things to do, in order to make vcimgnetclient.py Python3 compatible. Therefore, I implement my own vcimgnetclient_qt.py based on PyQt5.
2.3.3 Longer Vision’s vcimgnetclient_qt.py
2.3.3.1 Server
1 | 22:42:22[root@VC-Z] /home/user |
2.3.3.2 Client
Sorry, I’m NOT going to show my code, but the performance can be demonstrated as follows:
Kaggle Competition - APTOS 2019 Blindness Detection
Oh-my-god, kaggle competition about diabetis… Hmmm, I have something to do these days… It’s also a GOOD chance for me to learn some medical terms… In this blog, we’re going to try out this kaggle competition - APTOS 2019 Blindness Detection.
1. Pre-processing Diabetic Retinopathy Images
It’s NOT hard for us to locate APTOS [UpdatedV14] Preprocessing- Ben’s & Cropping, which seems to me a professional snippet of code preprocessing diabetic retinopathy images. Let’s just try it out directly.
1.1 Implementation
The implementation is cited directly from APTOS [UpdatedV14] Preprocessing- Ben’s & Cropping, with trivial modifications. You are welcome to download my snippet of code here.